Repository navigation
test(r13): evaluator matrix and clause-level corpus cases (R13 slice 1) - #83
Merged
Merged
Conversation
No engine change. tests/evaluator_matrix_tests.rs runs 36 predicates through both evaluators - RecursiveWhereEvaluator as WHERE, ArithmeticEvaluator as a SELECT item - over a five-row table with NULL in every operand position, and compares each row with the three-valued answer DuckDB gives for the same table. WHERE evaluator: 36/36. Value evaluator: 4/36. The 32 divergences are recorded in KNOWN, grouped by fault (comparison, NOT, IN, BETWEEN, LIKE, AND/OR), under the corpus contract: a recorded divergence that changes - including being fixed - fails the test, so slice 2 must update the file to prove what it moved. Six corpus cases pin the user-visible forms: SELECT `= NULL`, IN with NULL, AND with NULL (P48); HAVING NOT over an all-NULL group (P50); unparenthesised AND in a SELECT item (P51, GAP); a join on NULL keys on both sides (P52). Parity AGREE unchanged at 168; 197 -> 203 cases. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW
P48 now covers the whole value evaluator: six faults, including NULL = NULL comparing equal (P18, still live outside WHERE). P50 HAVING admits a group whose predicate is UNKNOWN under NOT. P51 AND/OR not accepted in a SELECT item unless parenthesised. P52 a join on NULL keys pairs NULL rows, on both the hash and nested-loop paths. R13 marked in progress with slice 1 done, its result, and the base decision: the value evaluator's shape carries the WHERE evaluator's rules. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
R13 slice 1: pin every divergence between the two expression evaluators before changing either. No engine change. Parity AGREE is unchanged at 168; cases go from 197 to 203.
The matrix
tests/evaluator_matrix_tests.rsruns 36 predicates through both evaluators:RecursiveWhereEvaluator, asWHERE <predicate>ArithmeticEvaluator, asSELECT (<predicate>)The table has five rows, with NULL in every operand position. Each row's answer is compared with what DuckDB returns for the same table; the expected strings were generated, not derived by hand.
The 32 divergences are recorded in
KNOWN, grouped by fault:to_boolIt uses the corpus contract. A recorded divergence that changes, including being fixed, fails the test (checked by breaking an entry on purpose). So slice 2 has to update this file to prove what it moved.
Corpus: the user-visible forms
select_eq_null_literal:score = NULLis TRUE for NULL rowsselect_in_list_with_nullselect_and_with_nullhaving_not_over_null: an all-NULL group is kept underNOTselect_and_unparenthesised:SELECT a AND bdoes not parsejoin_on_null_key_both_sides: 32 rows vs 7Docs
compare_with_op.KNOWNentry disappears, and the corpus moves by exactly the P48/P50 cases.🤖 Generated with Claude Code
https://claude.ai/code/session_01GpuTexUeuyuAdmDsukMTRW